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SUMMARY 


A model of a central processor (CPU) which services background applica- 
tions in the presence of time-critical activity is presented. The CPU is viewed 
as an M/M/1 queueing system subject to periodic interrupts by a deterministic, 
time-critical process. The Laplace transform of the distribution of service 
times for the background applications is developed. The use of state-of-the- 
art queueing models for studying the background processing capability of time- 
critical computer systems is discussed and the results of a model-validation 
study which support this application of queueing models are presented. 


INTRODUCTION 

A time-critical system is one in which the periodic and deterministic pro- 
cessing requirements of certain applications must be guaranteed in order that 
strict timing constraints can be met. The environment is said to be "hard real- 
time” since task deadlines are rigid. Unknown and untimely responses cannot be 
tolerated in such systems. 

Real-time digital and hybrid simulations of physical systems (refs. 1 to 3) 
are examples of time-critical applications. Because the time-critical integrity 
of these applications must be maintained, a portion of system resources (central 
memory, I/O processors, channels, and disks) generally must be fully dedicated 
to their use. State-of-the-art computer systems, however, allow the remaining 
resources to be used by background batch and interactive applications (ref. 4) . 
It is the performance of these systems when serving background applications that 
will be considered here. 

Queueing theory has been used to model many aspects of computer systems, 
for example, batch and interactive systems, paged memory systems, I/O subsys- 
tems, communication networks, and systems with mixed classes of applications 
(refs. 5 to 8). The basis for applying this theory to multiprogrammed computer 
systems is the assumption that the population of user tasks, as a whole, behaves 
in a stochastic manner. For the normal multiprogrammed environment, this is a 
reasonable assumption. However, the deterministic nature of a time-critical 
process violates the Poisson assumptions of most queueing models, and it is not 
immediately evident that a queueing model can reasonably be applied to the study 
of systems which operate in this environment. This application of queueing 
models is the subject of this paper. 
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pdf of X conditioned on Y 
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CPU MODEL 

The CPU (fig. 1) is viewed as a single-server queueing facility in which 
a high-priority, time-critical task has immediate access to the CPU and inter- 
rupts the servicing of background tasks. The arrival of background tasks is 
a Poisson process with an average arrival rate X. The required CPU service 
times for these tasks are assumed to be independent and identically distributed 
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Figure 1.- Time-critical CPU model. 


random variables from an exponential distribution with an expected service time 
p" 1 . The time-critical task is completely deterministic, unlike preemptive- 
priority queueing models or models in which the server is subject to breakdowns 
(refs. 9 and 10). This task arrives at a constant frame interval F and 
requires the CPU for a constant fraction £ of each frame. The time-critical 
process and the CPU server are combined to form a virtual server. The objective 
is to develop the resulting Laplace transform of virtual service time for the 
background tasks. 

The virtual service time random variable V includes the required CPU ser- 
vice time of the background task and the accumulated holding time that occurs 
while the time-critical task is being serviced. The initiation of virtual ser- 
vice is independent of the time-critical process. That is, whenever both the 
CPU and holding stage are free of a background task, then an arriving background 
task can proceed to the holding stage immediately. Now if the time-critical 
task is processing, then the background task remains at the holding stage; 
otherwise, the background task receives CPU service immediately and no holding 
time is incurred. 

During the first £F time units of each frame the time-critical task occu- 
pies the CPU, while for the remaining time (1 - £)F the CPU is available for 
background processing. Since this process is identical for each frame, the 
beginning of a frame (the point at which the interrupt occurs) is used as a 
point of reference. The random variable T, measured from this reference point, 
is the time into the frame that a background task begins to receive CPU service. 

It is easy to recognize the similarity between this model and a round-robin 
(RR) model, which includes an overhead penalty for exchanging tasks (for example, 
refs. 11 to 13). The RR system divides time into slices known as quantum inter- 
vals. Each task, in a cyclic fashion, is assigned a quantum interval and then 
exchanged for a new task at the end of the quantum. In practice some amount of 
overhead is required in each quantum to perform the task exchange. This over- 
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head is equivalent to the time-critical service time %F, whereas the quantum 
interval is equivalent to the frame interval F. The primary difference is 
that, for the model considered here, virtual service can begin at any point in 
the frame, i.e., 0 < T < F, whereas for the RR model T = 0 when the overhead 

is assumed at the beginning of a quantum. Additionally, each task in the cur- 
rent model is completed before service on another background task begins. 

Suppose that T = 0. If a background task is still processing at the end 
of the frame, then there is a probability a that additional service is 
required where 


a = Pr[v > (1 - £)F] = e“M F n _ Ji) 


( 1 ) 


Since the probability that additional service is required is independent of the 
amount of service received during past frames (the memoryless property of the 
exponential) , then the number of frames of service R is a random variable from 
the geometric probability density function (pdf) , 


Pr[R=r] = a r_1 0 - a) 


(r = 1, 2, 3, . . .) (2) 


During the frame in which a task departs from the system, the task will receive 
D units of CPU service, where D is from the truncated exponential pdf. 
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which has a Laplace transform D*(s) given 
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by 

e - (s+M) (1 — &) Fj 
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In general, however, T can occur in either of two intervals: the time- 
critical interval, defined as 0 S T < £F, or the background interval, defined 
as $,F < T < F. Consider first that virtual service begins during the back- 
ground interval. Let Egj represent this event. For a task beginning service 
in this interval there is an amount F - T of CPU time available in the first 
frame. If this is a sufficient amount of processing time, then V has the nor- 
malized exponential pdf 
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(0 < X < F - T) 


(5) 
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with the Laplace transform 


V*(s|E BI ,V<F-T) 


y (1 - e-( s -y) ( F - T )) 
(s + y) (1 — ) 


( 6 ) 


A task not completing service in this initial period will be interrupted by the 
time-critical process. In that case, the total virtual service time is the sum 
of the initial service F-T, an integral number of frames of interrupted ser- 
vice (R - 1 ) F, and the last frame of virtual service &F + D. Let 


X = (F-T) + (fl,F + D) (7) 


the pdf of which has the Laplace transform 

X* (s) = e -s (F-1H-£F) B * ( s ) (8) 

since F - T + &F is constant for a given T. Let 

Y = (R - 1 ) F (9) 

the pdf of which has the Laplace transform 


■ I 

r=l 


Y* (s) = ) a r “^ (1 - a)e -s ( r 1 ) F = 
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( 10 ) 


Since X and Y are independent random variables, then 


V*(s|E BI ,V>F-T) = X*(s) Y* (s) = 


y(l - e -(s+y) (l-fi,)F) ( e -s(F-TtjlF) ) 
(s + vi) (1 - ae” sF ) 


(11) 
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Using 


Pr[v>F-T] = e"l J ( F “ T ) 


( 12 ) 


and by the law of total probability, 


V*(s|E bi ) = V*(s|E bi ,V>F-T) Pr[v>F-T] + V* (s| E BI ,V<F-T) (l - Pr[v>F-T]) 


s + y 


1 + 


(e-sAF _ d (e -(s+y) (F-T) ) 
(1 - ae -sF ) 


( 13 ) 


Suppose that virtual service begins during the time-critical interval. Let 
this event be Erpj. In that case there will be an initial delay of £,F - T 
before the background interval begins. The virtual service time for this condi- 
tion is 


V = &F - T + (R - 1)F + D 


( 14 ) 


the pdf of which has the Laplace transform 


V*(s|E TI ) 


y (l _ e -(s+y) (1-£)F) ( e -s(5,F-T) j 
(S + y) (1 - ae“ sF ) 


05 ) 


Now consider the distribution of service time with regard to the number 
of background tasks that an arrival finds either queued or in virtual service. 
If an arrival finds no background tasks in the system, i.e., an idle virtual 
server, then the Laplace transform of virtual service will be either V*(s|E B j) 
or V* (s| Erjij) , depending on the point of arrival. Let Ej represent the idle 
virtual server condition. Removing the arrival-time condition, then 


V* (s 


r>S-F 

l F I> " J 

J t=0 


V*(s|E ti ) f T (t) dt + 


r»F 
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V*(s|E bi ) f T (t) dt 
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( 16 ) 
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where 


K S 


(1 - ae-BFd-fc)) (1 - e - sp ft) 

1 - ae” sF 


(17) 


and where 


f T (t) 


1 

F 


(18) 


because the distribution of interarrival times from a Poisson stream is known to 
be uniform. (See ref. 14, pp. 64 to 65, for a proof.) However, if the arrival 
finds a busy virtual server, denoted as the event Eg, then the virtual service 
of this arrival will begin at the departure point of the previous background 
task. Now since departures can only occur during the background interval, then 
an arrival to a busy server will always receive service from the distribution 
having a Laplace transform V*(s|E B j). Removing the service initiation-time 
condition results in 


V* (s 


|E B> - I 


Kc 


V*(s|Egj) frp(t|Egj) dt - 
't=£F s + y 


(1 - Z) (s + y) 2 


(19) 


where 


fT(t| E B i) 


1 

(1 - Z) F 


(20) 


because interdeparture times of an exponential distribution are uniformly dis- 
tributed and where K s is given by equation (17). 

Clearly then, the single-server queueing facility subjected to the inter- 
rupts of a time-critical process gives rise to a situation in which the first 
background arrival to initiate a busy period (having found an idle server) 
receives service from a distribution having a Laplace transform given by 
V* (s| Ej) , whereas all other arrivals receive service from a distribution having 
a Laplace transform given by V*(s|E B ). Such a generalization of the M/G/l 
queueing system has been studied by Welch (ref. 15). Welch shows that the 
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expected system response time E(W), which includes both queue wait time and 
service time, is given by 


E(W) = E(V) + 


XE(V 2 ) 

2[l - \E(V)](l - X[E(V|E B ) - E(V|E X )]} 


( 21 ) 


where E(V) and E(V 2 ) are the first and second moments of service time, 
respectively, and E(V|E B ) and E(V|E X ) are the expected values of the condi- 
tioned service times. Note that when E(V|E B ) = E(V|E X ) the well-known 
Pollaczek-Khinchin formula for the M/G/l queue is obtained. The Laplace trans- 
form of service time V*(s) is given by 


V* (s) = V*(s|E B ) Pr[E B ] + V*(s|E x )(l - Pr[E B ]) 


( 22 ) 


where 


?r[E B ] 


XE (V| E x ) 

1 - X[E(V|E B ) - E(V|E X )] 


(23) 


is the probability of finding a busy server. Using the transforms given by 
equations (16) and (19), then 


A dV* (s| E b ) 
E(V|E b ) = - 


ds 


,-o 


(24) 


Similarly, 


1 + e-| 

E(V|E X ) = (25) 

yd - Z) 


where 


il 2 QiF(1 - &) (1 + g) - 2(1 - g)] 
2(1 - a) 


( 26 ) 
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Thus, 


E (V) = — + AE(V) 

y(i - l) 


AE (V) = 


e1 (1 - P') 
y(i - ^) n + P'e-| ) 


y(i - S-) 


The second moment of virtual service time is obtained as follows; 


A d2 V *(s|EB) 

E (V 2 | E b ) = 

ds 


[no - 1 )] 2 


where e^ is given by equation (26). Similarly, 


e^Iej) = — + e 2 

[y o - S ')] 2 


where 


2yFS, 2 (l — fi,) 2 (1 + a) 2(yF) 2 Jl 2 (l - l) 2 a (yF) 2 2, 3 (l 
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1 - a (1 - a ) 2 3 

+ 42, 3 - 62, 2 
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Thus, 


E (V 2 ) = 


= E(V 2 |E b ) Pr[E B ] + E (V 2 | Ej) ^1 - Pr[E B ]j = - 


P d - £)] 


+ AE (V 2 ) 


(33) 


where 


AE(V 2 ) = 


e 2 + p' (1 + ei) (2ei - e 2 ) (1 + p'e-|) 

[yd - *)] 2 


(34) 


Finally, from equation (21), the expected system response time is 


1 

E(W) = + AE(W) 

yd - l) (l - P') 


(35) 


where 


X AE(V 2 ) 

AE(W) = AE (V) + (36) 

2(1 - p') 


TIME-CRITICAL CPU ALLOCATION 

The variables for this model are set by the requirements of the background 
workload and the time-critical process for a particular CPU and are presumably 
fixed. However, there is some flexibility in the allocation of CPU service to 
the time-critical task. In a practical sense, this allocation is constrained by 
the time in which results of computation must be available for transfer to out- 
put devices, i.e., the task deadline. It shall be assumed here that the dead- 
line time is the same as the frame time, so that once an interrupt occurs there 
is no constraint on time-critical CPU allocation as long as JlF units of pro- 
cessing time are realized by the process during each frame. We may consider, 
then, subdividing F into n equal quantum intervals of length Q = F/n and 
in each of these quantums allocating ZQ units of processing time for the time- 
critical process and the remaining time for background processing. Also assume 
now that a fixed overhead time <J> is incurred during each quantum due to the 
alternation of service between the time-critical task and the background task. 
This overhead is expressed as some fractional portion of the frame, i.e., 

^ = <{>/F. 
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The effect of this action is that the time-critical load is more uniformly 
distributed over the frame, thus changing the characteristics of virtual ser- 
vice. The effects of this load-leveling action on the system response time can 
be calculated by replacing P with P/n and by adding the overhead nJU, where 
n &<J> = 4 </Q» to the required time-critical load £,. For example, suppose 
&<}> = 0.005, l = 0.500, and p = X/y = 0.200. The percentage improvement in 
response time due to load leveling, defined as 


Response- time 


improvement = 


100[E(W|n=l) - E(W|n)] 
E(W|n=l) 


(n = 1, 2, 3, . . .) (37) 


is shown in figure 2 for several values of the product yP. 



Figure 2.- Effect of time-critical load leveling on 
response time. Jl = 0.5; p = 0.2; JU = 0.005. 
(N = n in the text.) 


It is apparent that for smaller values of yF the overhead associated with 
task switching is the dominant factor and immediately degrades system perform- 
ance. However, when the time-critical frame is large compared with the average 
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CPU service time for background tasks (large yF) , then load leveling (to a 
point) would indeed improve system performance. This model should prove help- 
ful in examining that proposition for a given system. Note that, for this par- 
ticular set of parameters and for yF equal to 4 or 10, simply dividing F 
into two equal segments produces nearly optimal results. 


APPLICABILITY OF STATE-OF- THE- ART QUEUEING MODELS 

Consider now the feasibility of using existing queueing models to study 
time-critical systems. One advantage of this capability is the fact that soft- 
ware tools (refs. 7, 16, and 17) have been developed for these models which 
greatly simplify the process of model development and analysis. The model of 
Baskett, Chandy, Muntz, and Palacios (ref. 18) is a summarization and generali- 
zation of the results of many researchers and is representative of the state 
of the art in applying queueing theory to computer systems. This model allows 
one to distinguish between classes of applications, for example, batch and 
interactive, and can be used to study open (exogenous arrivals), closed (no 
exogenous arrivals) , and mixed (closed with respect to some classes and open 
to other classes) networks of queueing facilities. Four different types of 
service facilities can be used to represent the various computer resources 
(CPU's, disks, terminals, etc.). The service distributions for these resources 
need only have rational Laplace transforms. If scheduling is first come, first 
served, however, the distribution must also be exponential. While this is a 
very general model, it does not provide a solution for preemptive— resume prior- 
ity scheduling. Thus, if the time-critical process is modeled explicitly as a 
separate customer class, then this model is not appropriate, since preemptive- 
resume priority scheduling would be required. (Approximation techniques, how- 
ever, have been developed for this scheduling discipline and have been used in 
some models, e.g., ref. 7.) 

However, for the class of computer system considered here, it is proposed 
that it is not necessary to explicitly represent the time-critical task in order 
to study the performance of the background-processing system. As previously 
mentioned, because of stringent timing constraints, it is generally necessary 
to totally dedicate a portion of system resources to the time-critical appli- 
cations. Such a system can be modeled as shown in figure 3. Conceptually the 
system is viewed as two machines: a background machine and a time— critical 
machine, each sharing the CPU but otherwise using mutually exclusive sets of 
resources. The performance of the time-critical machine is deterministic and 
the objective is to model the background machine. 

The alternative to an explicit representation of the time-critical task 
is to implicitly include the effects of this task on the performance of the 
background machine by using the virtual service time distribution developed 
previously. However, when an exponential server is subjected to a time- 
critical load, the resulting service distribution is not exponential, nor is 
its Laplace transform (eq. (22)) rational. An approximation for this distri- 
bution is in order. Again, consider the magnitude of the product yF. A small 
product represents the average CPU service time for background tasks taking 
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Figure 3.- A time-critical system with a background- 
processing capability. 


place over many frames or, equivalently, a small F compared with the required 
CPU service time for background tasks. Referring to equations (26) and (32) , 
it can be shown that 


lira pP*0 e l = 0 


and 


linipp+o e 2 = 0 


so that AE(V) in equation (28) and AE(V 2 ) in equation (33) will also 
approach zero as the frame becomes small compared with the required CPU service 
time. The result is that for small pF, an exponential server with parameter 
p, operating under a time-critical load J£, appears to the input stream as an 
exponential server with parameter p (1 — Si) • Thus, two independent processors 
emerge: a time-critical processor with a service rate of £ times the actual 

CPU service rate, and a background processor with a service rate of 1 - £ 
times the actual CPU service rate. The total CPU utilization is the weighted 
utilization of each of these processors, that is, 


CPU utilization = (1 - JUUq + jlU^ 


(38) 
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where U B and 1%; are the utilization of the background and time-critical 
processors, respectively. Note that if S is the service rate of the actual 
CPU (used a fraction % of each frame by the time-critical task) , then the 
time-critical processor with a service rate of S& must be utilized 100 per- 
cent, so that U>pc = 1 . Note also that the actual background CPU utilization is 
obtained by multiplying the background processor utilization U B (obtained from 
a queueing model) by the factor 1 - i. 

This approximation is intuitively appealing and immediately offers a means 
for making use of existing queueing models. That is, one assumes an exponential 
server with the average service rate modified by the time-critical load. The 
accuracy of this approximation to the service distribution can be examined by 
forming the following: 


100 AE(V) 

First moment error = — ; (39) 

VB> <’ - *>] 

100 AE(V 2 ) 

Second moment error = r (40) 

2/[y(l - SL)y 

These are shown first as a function of fi, and p' for pF = 1 in figures 4 
and 5. It is found that the maximum errors occur at the midregion of Then, 

for i = 0.5, the above errors are shown in figures 6 and 7 as a function of 
pF for several values of p'. These latter figures, then, show the maximum 
errors that can be expected for a given value of the product pF as a result 
of the approximate service distribution. Clearly, the magnitude of the product 
pF is a good indicator of the accuracy of the approximation. 
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EXPERIMENTAL RESULTS 

The results of the previous section were applied to a performance evalua- 
tion study of a digital simulation facility located at the NASA Langley Research 
Center in Hampton, Virginia (ref, 19). This facility, is capable of supporting 
batch, interactive, and time-critical applications. However, in order to run 
controlled experiments, only the batch and time-critical capabilities were used. 
System measurements determined that the average batch CPU service time was 
12.20 msec. Because the majority of simulation applications used a frame time 
of 31.25 msec, the resulting yF of 2.6 indicated that the average CPU service 
rate for a time-critical load & could reasonably be formulated as 


yOl) = 12. 2" 1 (1 - H) 


(41) 


A closed queueing model was developed for the batch processing system. 
Controlled experiments were made in a purely batch environment in order to cal- 
ibrate and validate the model. Then a time-critical task was introduced into 
the system and the experiments were repeated. The CPU service rate of equa- 
tion (41) was used and degree of multiprogramming of the model (average number 
of circulating batch tasks) was modified to account for the dedicated use of 
central memory by the time-critical task. A comparison of measured results and 
model results for CPU utilization and CPU queue length is shown in the follow- 
ing table. As can be seen, once a calibrated batch model was obtained, the 
effects of the time-critical task were accurately predicted by the model. 
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Time-critical 
CPU utilization 

i, % 

Batch CPU utilization, % 

Average CPU queue length 

(1 - A) x model 

Measured 

Model 

Measured 

0 (Calibration) 

60.9 

60.5 

1.5 

1.3 

10.0 

59.7 

59.3 

1.7 

1.6 

30.0 

54.4 

53.8 

2.6 

2.2 

50.0 

44.7 

45.0 

3.9 

3.5 

70.0 

29.5 

29.3 

5.8 

5.5 


CONCLUDING REMARKS 

The Laplace transform of the service distribution for an M/ty/1 queueing 
system subject to the periodic interrupts of a deterministic time-critical pro- 
cess has been developed. By using this distribution, the benefits of uniformly 
distributing the load of the time-critical process can be evaluated and weighed 
against the increased overhead that is incurred due to the alternation of ser- 
vice between the time-critical task and the background task. 

The Laplace transform of this distribution does not have a rational form, 
so state-of-the-art network queueing models are not directly applicable. How- 
ever, if the time between interrupts is small compared with the average CPU ser- 
vice time of the background tasks, then an approximate service distribution is 
an exponential distribution, with the original service rate multiplied by the 
fraction of processing time available to background tasks. This suggests that 
for an appropriate range of model parameters, current queueing models can rea- 
sonably be extended to the study of background processing in time-critical 
environments. Experimental results are reported which support this application. 


Langley Research Center 

National Aeronautics and Space Administration 
Hampton, VA 23665 
September 14, 1979 
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